{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Emotion Classification using TAO EmotionNet\n",
    "\n",
    "Transfer learning is the process of transferring learned features from one application to another. It is a commonly used training technique where you use a model trained on one task and re-train to use it on a different task. \n",
    "\n",
    "Train Adapt Optimize (TAO) Toolkit is a simple and easy-to-use Python based AI toolkit for taking purpose-built AI models and customizing them with users' own data.\n",
    "\n",
    "<img align=\"center\" src=\"https://d29g4g2dyqv443.cloudfront.net/sites/default/files/akamai/TAO/tlt-tao-toolkit-bring-your-own-model-diagram.png\" width=\"1080\"> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning Objectives\n",
    "In this notebook, you will learn how to leverage the simplicity and convenience of TAO to:\n",
    "\n",
    "* Take a pretrained model and train an EmotionNet model on subset of CK+ dataset\n",
    "* Run Inference on the trained model\n",
    "* Export the retrained model to a .etlt file for deployment for DeepStream SDK\n",
    "\n",
    "At the end of this notebook, you will have generated a trained and optimized `EmotionNet` model\n",
    "which you may deploy via [DeepStream](https://developer.nvidia.com/deepstream-sdk).\n",
    "\n",
    "### Table of Contents\n",
    "\n",
    "This notebook shows an example of emotion classification in the Train Adapt Optimize (TAO) Toolkit.\n",
    "\n",
    "0. [Set up env variables, map drives, and install dependencies](#head-0)\n",
    "1. [Install the TAO launcher](#head-1)\n",
    "2. [Prepare dataset and pre-trained model](#head-2) <br>\n",
    "    2.1 [Verify downloaded dataset](#head-2-1) <br>\n",
    "    2.2 [Convert dataset labels to required json format](#head-2-2) <br>\n",
    "    2.3 [Verify dataset conversion](#head-2-3) <br>\n",
    "    2.4 [Download pre-trained model](#head-2-4) <br>\n",
    "3. [Generate tfrecords from labels in json format](#head-3)\n",
    "4. [Provide training specification](#head-4)\n",
    "5. [Run TAO training](#head-5)\n",
    "6. [Evaluate trained models](#head-6)\n",
    "7. [Run TAO inference](#head-7)\n",
    "8. [Deploy](#head-8)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0. Set up env variables, map drives and install dependencies <a class=\"anchor\" id=\"head-0\"></a>\n",
    "When using the purpose-built pretrained models from NGC, please make sure to set the `$KEY` environment variable to the key as mentioned in the model overview. Failing to do so, can lead to errors when trying to load them as pretrained models.\n",
    "\n",
    "The following notebook requires the user to set an env variable called the `$LOCAL_PROJECT_DIR` as the path to the users' workspace. Please note that the dataset to run this notebook is expected to reside in the `$LOCAL_PROJECT_DIR/emotionnet/data`, while the TAO experiment generated collaterals will be output to `$LOCAL_PROJECT_DIR/emotionnet`. More information on how to set up the dataset and the supported steps in the TAO workflow are provided in the subsequent cells.\n",
    "\n",
    "*Note: This notebook currently is by default set up to run training using 1 GPU. To use more GPU's please update the env variable `$NUM_GPUS` accordingly*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setting up env variables for cleaner command line commands.\n",
    "import os\n",
    "\n",
    "%env KEY=nvidia_tlt\n",
    "%env NUM_GPUS=1\n",
    "%env USER_EXPERIMENT_DIR=/workspace/tao-experiments/emotionnet\n",
    "%env DATA_DOWNLOAD_DIR=/workspace/tao-experiments/emotionnet/orgData\n",
    "\n",
    "# Set this path if you don't run the notebook from the samples directory.\n",
    "# %env NOTEBOOK_ROOT=~/tao-samples/emotionnet\n",
    "\n",
    "# Please define this local project directory that needs to be mapped to the TAO docker session.\n",
    "# The dataset is expected to be present in $LOCAL_PROJECT_DIR/emotionnet/orgData, while the results for the steps\n",
    "# in this notebook will be stored at $LOCAL_PROJECT_DIR/emotionnet\n",
    "# !PLEASE MAKE SURE TO UPDATE THIS PATH!.\n",
    "%env LOCAL_PROJECT_DIR=FIXME\n",
    "\n",
    "# $SAMPLES_DIR is the path to the sample notebook folder and the dependency folder\n",
    "# $SAMPLES_DIR/deps should exist for dependency installation\n",
    "%env SAMPLES_DIR=FIXME\n",
    "\n",
    "os.environ[\"LOCAL_DATA_DIR\"] = os.path.join(\n",
    "    os.getenv(\"LOCAL_PROJECT_DIR\", os.getcwd()),\n",
    "    \"emotionnet/orgData\"\n",
    ")\n",
    "os.environ[\"LOCAL_EXPERIMENT_DIR\"] = os.path.join(\n",
    "    os.getenv(\"LOCAL_PROJECT_DIR\", os.getcwd()),\n",
    "    \"emotionnet\"\n",
    ")\n",
    "\n",
    "# The sample spec files are present in the same path as the downloaded samples.\n",
    "os.environ[\"LOCAL_SPECS_DIR\"] = os.path.join(\n",
    "    os.getenv(\"NOTEBOOK_ROOT\", os.getcwd()),\n",
    "    \"specs\"\n",
    ")\n",
    "\n",
    "# The sample spec files are present in the same path as the downloaded samples.\n",
    "os.environ[\"LOCAL_DATASET_SPECS_DIR\"] = os.path.join(\n",
    "    os.getenv(\"NOTEBOOK_ROOT\", os.getcwd()),\n",
    "    \"dataset_specs\"\n",
    ")\n",
    "\n",
    "%env SPECS_DIR=/workspace/tao-experiments/emotionnet/specs\n",
    "%env DATASET_SPECS_DIR=/workspace/tao-experiments/emotionnet/dataset_specs\n",
    "\n",
    "# Showing list of specification files.\n",
    "!ls -rlt $LOCAL_SPECS_DIR\n",
    "!ls -rlt $LOCAL_DATASET_SPECS_DIR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The cell below maps the project directory on your local host to a workspace directory in the TAO docker instance, so that the data and the results are mapped from in and out of the docker. For more information please refer to the [launcher instance](https://docs.nvidia.com/tao/tao-toolkit/text/tao_launcher.html) in the user guide.\n",
    "\n",
    "When running this cell on AWS, update the drive_map entry with the dictionary defined below, so that you don't have permission issues when writing data into folders created by the TAO docker.\n",
    "\n",
    "```json\n",
    "drive_map = {\n",
    "    \"Mounts\": [\n",
    "            # Mapping the data directory\n",
    "            {\n",
    "                \"source\": os.environ[\"LOCAL_PROJECT_DIR\"],\n",
    "                \"destination\": \"/workspace/tao-experiments\"\n",
    "            },\n",
    "            # Mapping the specs directory.\n",
    "            {\n",
    "                \"source\": os.environ[\"LOCAL_SPECS_DIR\"],\n",
    "                \"destination\": os.environ[\"SPECS_DIR\"]\n",
    "            },\n",
    "            # Mapping the dataset specs directory.\n",
    "            {\n",
    "                \"source\": os.environ[\"LOCAL_DATASET_SPECS_DIR\"],\n",
    "                \"destination\": os.environ[\"DATASET_SPECS_DIR\"]\n",
    "            },\n",
    "        ],\n",
    "    \"DockerOptions\": {\n",
    "        \"user\": \"{}:{}\".format(os.getuid(), os.getgid())\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Mapping up the local directories to the TAO docker.\n",
    "import json\n",
    "mounts_file = os.path.expanduser(\"~/.tao_mounts.json\")\n",
    "\n",
    "# Define the dictionary with the mapped drives\n",
    "drive_map = {\n",
    "    \"Mounts\": [\n",
    "        # Mapping the data directory\n",
    "        {\n",
    "            \"source\": os.environ[\"LOCAL_PROJECT_DIR\"],\n",
    "            \"destination\": \"/workspace/tao-experiments\"\n",
    "        },\n",
    "        # Mapping the specs directory.\n",
    "        {\n",
    "            \"source\": os.environ[\"LOCAL_SPECS_DIR\"],\n",
    "            \"destination\": os.environ[\"SPECS_DIR\"]\n",
    "        },\n",
    "        # Mapping the dataset specs directory.\n",
    "        {\n",
    "            \"source\": os.environ[\"LOCAL_DATASET_SPECS_DIR\"],\n",
    "            \"destination\": os.environ[\"DATASET_SPECS_DIR\"]\n",
    "        },\n",
    "    ]\n",
    "}\n",
    "\n",
    "# Writing the mounts file.\n",
    "with open(mounts_file, \"w\") as mfile:\n",
    "    json.dump(drive_map, mfile, indent=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cat ~/.tao_mounts.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install requirement\n",
    "!pip3 install Cython==0.29.36\n",
    "!pip3 install -r $SAMPLES_DIR/deps/requirements-pip.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Install the TAO launcher <a class=\"anchor\" id=\"head-1\"></a>\n",
    "The TAO launcher is a python package distributed as a python wheel listed in PyPI. You may install the launcher by executing the following cell.\n",
    "\n",
    "Please note that TAO Toolkit recommends users run the TAO launcher in a virtual env with python 3.6.9. You may follow the instruction on this [page](https://virtualenvwrapper.readthedocs.io/en/latest/install.html) to set up a python virtual env using the `virtualenv` and `virtualenvwrapper` packages. Once you have set up virtualenvwrapper, please set the version of python to be used in the virtual env by using the `VIRTUALENVWRAPPER_PYTHON` variable. You may do so by running\n",
    "\n",
    "```sh\n",
    "export VIRTUALENVWRAPPER_PYTHON=/path/to/bin/python3.x\n",
    "```\n",
    "where x >= 6 and <= 8\n",
    "\n",
    "We recommend performing this step first and then launching the notebook from the virtual environment. In addition to installing TAO python package, please make sure of the following software requirements:\n",
    "* python >=3.7, <=3.10.x\n",
    "* docker-ce > 19.03.5\n",
    "* docker-API 1.40\n",
    "* nvidia-container-toolkit > 1.3.0-1\n",
    "* nvidia-container-runtime > 3.4.0-1\n",
    "* nvidia-docker2 > 2.5.0-1\n",
    "* nvidia-driver > 455+\n",
    "\n",
    "Once you have installed the pre-requisites, please log in to the docker registry nvcr.io by following the command below\n",
    "\n",
    "```sh\n",
    "docker login nvcr.io\n",
    "```\n",
    "\n",
    "You will be triggered to enter a username and password. The username is `$oauthtoken` and the password is the API key generated from `ngc.nvidia.com`. Please follow the instructions in the [NGC setup guide](https://docs.nvidia.com/ngc/ngc-overview/index.html#generating-api-key) to generate your own API key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Please replace fixme with the path to the wheel file that you downloaded from the developer zone link mentioned above.\n",
    "# SKIP this step IF you have already installed the TAO launcher wheel.\n",
    "!pip3 install nvidia-tao"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize the TAO launcher\n",
    "!tao info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Prepare dataset and pre-trained model <a class=\"anchor\" id=\"head-2\"></a>\n",
    "\n",
    "Please download the CK+ dataset from: https://www.pitt.edu/~emotion/ck-spread.htm.\n",
    "You will need to sign the dataset user agreement and send it to the email provided on the agreement sheet to get access to the dataset.\n",
    "\n",
    "After obtaining the dataset, please place the files in `$DATA_DOWNLOAD_DIR`. Please rename the dataset folder to `ckplus` as `+` sign may not be a valid folder name.\n",
    "You will then have the following path for the CK+ dataset.\n",
    "* Input data in `$LOCAL_DATA_DIR/ckplus`\n",
    "\n",
    "You will then unzip the folder of ckplus dataset to the following folders.\n",
    "* Image data: `$LOCAL_DATA_DIR/ckplus/cohn-kanade-images`\n",
    "* Emotion label data: `$LOCAL_DATA_DIR/ckplus/Emotion`\n",
    "* Landmarks label data: `$LOCAL_DATA_DIR/ckplus/Landmarks`\n",
    "\n",
    "Note: please make sure that the folder name are as listed above. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Verify downloaded dataset <a class=\"anchor\" id=\"head-2-1\"></a>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check the dataset is present\n",
    "!mkdir -p $LOCAL_DATA_DIR\n",
    "!if [ ! -d $LOCAL_DATA_DIR/ckplus/cohn-kanade-images ]; then echo 'Image Data folder not found, please download.'; else echo 'Found Image Data folder.';fi\n",
    "!if [ ! -d $LOCAL_DATA_DIR/ckplus/Emotion ]; then echo 'Emotion labels folder not found, please download.'; else echo 'Found Emotion Labels folder.';fi\n",
    "!if [ ! -d $LOCAL_DATA_DIR/ckplus/Landmarks ]; then echo 'Landmarks labels folder not found, please download.'; else echo 'Found Landmarks Labels folder.';fi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B. Convert dataset labels to required json format <a class=\"anchor\" id=\"head-2-2\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 ckplus_convert.py --root_path $LOCAL_EXPERIMENT_DIR --dataset_folder_name ckplus --container_root_path $USER_EXPERIMENT_DIR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C. Verify dataset conversion <a class=\"anchor\" id=\"head-2-3\"></a>\n",
    "\n",
    "Please use the provided conversion script `ckplus_convert.py` to convert existing `Landmarks` and `Emotion` labels from `CK+` dataset to the required json label format. \n",
    "\n",
    "Note: for other public datasets, please use this script as a reference to convert the labels to required format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sample json label.\n",
    "!sed -n 1,201p $LOCAL_DATA_DIR/ckplus/data_factory/fiducial/S052_004_00000031_happy.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### D. Download pre-trained model <a class=\"anchor\" id=\"head-2-4\"></a>\n",
    "\n",
    "Please follow the instructions in the following to download and verify the pretrain model for emotionnet.\n",
    "\n",
    "For EmotionNet pretrain model please download model: `nvidia/tao/emotionnet:trainable_v1.0`.\n",
    "\n",
    "After downloading the pre-trained model, please place the files in `$LOCAL_EXPERIMENT_DIR/pretrain_models`\n",
    "You will then have the following path\n",
    "\n",
    "* pretrain model in `$LOCAL_EXPERIMENT_DIR/pretrain_models/emotionnet_vtrainable_v1.0/model.tlt`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Installing NGC CLI on the local machine.\n",
    "## Download and install\n",
    "%env CLI=ngccli_cat_linux.zip\n",
    "!mkdir -p $LOCAL_PROJECT_DIR/ngccli\n",
    "\n",
    "# Remove any previously existing CLI installations\n",
    "!rm -rf $LOCAL_PROJECT_DIR/ngccli/*\n",
    "!wget \"https://ngc.nvidia.com/downloads/$CLI\" -P $LOCAL_PROJECT_DIR/ngccli\n",
    "!unzip -u \"$LOCAL_PROJECT_DIR/ngccli/$CLI\" -d $LOCAL_PROJECT_DIR/ngccli/\n",
    "!rm $LOCAL_PROJECT_DIR/ngccli/*.zip \n",
    "os.environ[\"PATH\"]=\"{}/ngccli/ngc-cli:{}\".format(os.getenv(\"LOCAL_PROJECT_DIR\", \"\"), os.getenv(\"PATH\", \"\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List models available in the model registry.\n",
    "!ngc registry model list nvidia/tao/emotionnet:*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the target destination to download the model.\n",
    "!mkdir -p $LOCAL_EXPERIMENT_DIR/pretrain_models/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download the pretrained model from NGC\n",
    "!ngc registry model download-version nvidia/tao/emotionnet:trainable_v1.0 \\\n",
    "    --dest $LOCAL_EXPERIMENT_DIR/pretrain_models/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls -rlt $LOCAL_EXPERIMENT_DIR/pretrain_models/emotionnet_vtrainable_v1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check the dataset is present\n",
    "!if [ ! -f $LOCAL_EXPERIMENT_DIR/pretrain_models/emotionnet_vtrainable_v1.0/model.tlt ]; then echo 'Pretrain model file not found, please download.'; else echo 'Found Pretrain model file.';fi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Generate tfrecords from labels in json format <a class=\"anchor\" id=\"head-3\"></a>\n",
    "* Create the tfrecords using the dataset_convert command\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!tao model emotionnet dataset_convert -c $DATASET_SPECS_DIR/dataio_config_ckplus.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check the result folder is present\n",
    "!mkdir -p $LOCAL_EXPERIMENT_DIR\n",
    "!if [ ! -d $LOCAL_EXPERIMENT_DIR/postData/ckplus/Ground_Truth_DataFactory ]; then echo 'Ground truth folder not found.'; else echo 'Found Ground truth folder.';fi\n",
    "!if [ ! -d $LOCAL_EXPERIMENT_DIR/postData/ckplus/GT_user_json ]; then echo 'GT user json folder not found.'; else echo 'Found GT user json folder.';fi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Provide training specification <a class=\"anchor\" id=\"head-4\"></a>\n",
    "* Tfrecords for the train datasets\n",
    "    * In order to use the newly generated tfrecords for training, update the 'ground_truth_folder_name' and 'tfrecords_directory_path' parameters of 'dataset_info' section in the spec file at `$LOCAL_SPECS_DIR/emotionnet_tlt_pretrain.yaml`\n",
    "* Pre-trained model path\n",
    "    * Update \"pretrained_model_path\" in the spec file at `$LOCAL_SPECS_DIR/emotionnet_tlt_pretrain.yaml`\n",
    "    * If you want to training from random weights with your own data, you can enter \"null\" for \"pretrained_model_path\" section\n",
    "* Augmentation parameters for on the fly data augmentation\n",
    "* Other training (hyper-)parameters such as batch size, number of epochs, learning rate etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cat $LOCAL_SPECS_DIR/emotionnet_tlt_pretrain.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Run TAO training <a class=\"anchor\" id=\"head-5\"></a>\n",
    "* Provide the sample spec file and the output directory location for models\n",
    "\n",
    "*Note: The training may take hours to complete. Also, the remaining notebook, assumes that the training was done in single-GPU mode. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!tao model emotionnet train -e $SPECS_DIR/emotionnet_tlt_pretrain.yaml \\\n",
    "                      -r $USER_EXPERIMENT_DIR/experiment_result/exp1 \\\n",
    "                      -k $KEY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls -lh $LOCAL_EXPERIMENT_DIR/experiment_result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Evaluate the trained model <a class=\"anchor\" id=\"head-6\"></a>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!tao model emotionnet evaluate -m $USER_EXPERIMENT_DIR/experiment_result/exp1/model.tlt \\\n",
    "                         -r $USER_EXPERIMENT_DIR/experiment_result/exp1 \\\n",
    "                         -e $SPECS_DIR/emotionnet_tlt_pretrain.yaml \\\n",
    "                         -k $KEY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check the Evaluation result file and summary file and is presented\n",
    "!if [ ! -f $LOCAL_EXPERIMENT_DIR/experiment_result/exp1/eval_results.txt ]; then echo 'Evaluation result summary file not found, please generate.'; else echo 'Found Evaluation result summary file.';fi\n",
    "!if [ ! -f $LOCAL_EXPERIMENT_DIR/experiment_result/exp1/full_results.txt ]; then echo 'Evaluation result file not found, please generate.'; else echo 'Found Evaluation result file.';fi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Visualize Inference <a class=\"anchor\" id=\"head-7\"></a>\n",
    "\n",
    "In this section, we run the inference tool to generate inferences on the trained models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Running inference for detection on n images\n",
    "!tao model emotionnet inference -e $SPECS_DIR/emotionnet_tlt_pretrain.yaml \\\n",
    "                          -i $DATA_DOWNLOAD_DIR/ckplus/data_factory/fiducial/S111_001_00000013_surprise.json \\\n",
    "                          -m $USER_EXPERIMENT_DIR/experiment_result/exp1/model.tlt \\\n",
    "                          -o $USER_EXPERIMENT_DIR \\\n",
    "                          -k $KEY "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!sed -n 1,1p $LOCAL_EXPERIMENT_DIR/result.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Simple emotion inference overlay visualizer\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import cv2\n",
    "import os\n",
    "import IPython.display\n",
    "import PIL.Image\n",
    "\n",
    "\n",
    "def visualize_images(output_path, num_images=10):\n",
    "    root_path = os.path.join(os.environ['LOCAL_EXPERIMENT_DIR'], output_path)\n",
    "    result_path = os.path.join(root_path, 'result.txt')\n",
    "    f = open(result_path, 'r')\n",
    "    lines = f.readlines()\n",
    "    \n",
    "    for k in range(0, num_images):\n",
    "        line = lines[k]\n",
    "        content = line.split(' ')\n",
    "        docker_data_path = os.environ['DATA_DOWNLOAD_DIR']\n",
    "        print(docker_data_path)\n",
    "        path_info = content[0].split(str(docker_data_path))\n",
    "        image_path = os.environ['LOCAL_DATA_DIR'] + path_info[-1]\n",
    "        emotion_label = content[1]\n",
    "        print(image_path, emotion_label)\n",
    "        img = cv2.imread(image_path, 0)\n",
    "        cv2.putText(img, \"Emotion: \" + emotion_label, (40, 40), 0, 1, 255)\n",
    "        IPython.display.display(PIL.Image.fromarray(img))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualizing the first 2 images.\n",
    "OUTPUT_PATH = '' # relative path from $USER_EXPERIMENT_DIR.\n",
    "IMAGES = 1\n",
    "visualize_images(OUTPUT_PATH, num_images=IMAGES)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Deploy <a class=\"anchor\" id=\"head-7\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p $LOCAL_EXPERIMENT_DIR/experiment_dir_final\n",
    "# Removing a pre-existing copy of the etlt if there has been any.\n",
    "import os\n",
    "output_file=os.path.join(os.environ['LOCAL_EXPERIMENT_DIR'],\n",
    "                         \"experiment_dir_final/emotionnet_onnx.etlt\")\n",
    "if os.path.exists(output_file):\n",
    "    os.system(\"rm {}\".format(output_file))\n",
    "!tao model emotionnet export -m $USER_EXPERIMENT_DIR/experiment_result/exp1/model.tlt \\\n",
    "                       -o $USER_EXPERIMENT_DIR/experiment_dir_final/emotionnet_onnx.etlt \\\n",
    "                       -t tfonnx \\\n",
    "                       -k $KEY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check the Deployed file is presented\n",
    "!if [ ! -f $LOCAL_EXPERIMENT_DIR/experiment_dir_final/emotionnet_onnx.etlt ]; then echo 'Deployed file not found, please generate.'; else echo 'Found Deployed file folder.';fi"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.10 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "vscode": {
   "interpreter": {
    "hash": "767d51c1340bd893661ea55ea3124f6de3c7a262a8b4abca0554b478b1e2ff90"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
